{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sigma example notebook\n",
    "\n",
    "This notebook should showcase some use cases with regards to Sigma."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -q timesketch_api_client\n",
    "!pip install -q timesketch_import_client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from timesketch_api_client import client as timesketch_client\n",
    "\n",
    "import requests\n",
    "import io\n",
    "import altair as alt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from timesketch_api_client import config\n",
    "from timesketch_import_client import helper\n",
    "from timesketch_import_client import importer\n",
    "from timesketch_api_client import search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create the sketch and import some data\n",
    "\n",
    "First we want to create a sketch and import a small data set. It is recommended to do that with the dev docker container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts_client = config.get_client()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create a new sketch\n",
    "\n",
    "That will be called test and will be filled with some test data from the Github repository.\n",
    "\n",
    "The downloaded csv will be read into a pandas dataframe for further processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sketch = ts_client.create_sketch(name=\"test\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://raw.githubusercontent.com/google/timesketch/master/test_tools/test_events/sigma_events.csv\" \n",
    "download = requests.get(url).content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(io.StringIO(download.decode('utf-8')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import the data to timesketch\n",
    "\n",
    "The easiest way to do that is using the import client as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import_helper = helper.ImportHelper() \n",
    "\n",
    "with importer.ImportStreamer() as streamer:\n",
    "  streamer.set_sketch(sketch)\n",
    "  streamer.set_config_helper(import_helper) \n",
    "\n",
    "  streamer.set_timeline_name('sigma_events.csv')\n",
    "  streamer.add_data_frame(df)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fetch Rules\n",
    "\n",
    "Now we want to fetch all the rules installed on the Timesketch instance. For the dev docker container we only expect one rule.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rules = ts_client.list_sigmarules()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for rule in rules:\n",
    "    print(f'ID: {rule.id} Title: {rule.title}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rule1 = rules[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The API also gives us the elastic search query we could use to explore the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rule1.search_query"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create a rule\n",
    "\n",
    "Lets create a single rule from a string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ZENMAPSIGMARULE = \"\"\"\n",
    "title: Suspicious Installation of Zenmap\n",
    "id: 5266a592-b793-11ea-b3de-0242ac130004\n",
    "description: Detects suspicious installation of Zenmap\n",
    "references:\n",
    "    - https://rmusser.net/docs/ATT&CK-Stuff/ATT&CK/Discovery.html\n",
    "author: Alexander Jaeger\n",
    "date: 2020/06/26\n",
    "modified: 2021/01/01\n",
    "logsource:\n",
    "    product: linux\n",
    "    service: shell\n",
    "detection:\n",
    "    keywords:\n",
    "        # Generic suspicious commands\n",
    "        - '*apt-get install zmap*'\n",
    "    condition: keywords\n",
    "falsepositives:\n",
    "    - Unknown\n",
    "level: high\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts_client.create_sigmarule(ZENMAPSIGMARULE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Single rule\n",
    "\n",
    "Next we want to fetch a single rule by a given uuid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "single_rule = ts_client.get_sigmarule(rule_uuid='5266a592-b793-11ea-b3de-0242ac130004')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "single_rule.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "single_rule.search_query"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Sigma examples\n",
    "\n",
    "The API also provides the option to provide a Sigma rule by text and the backend will parse it with the installed Sigma mappings. This can be especially helpful when developing new rules before installing and exposing them to all the other user of a Timesketch instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MOCK_SIGMA_RULE = r\"\"\"\n",
    "title: Suspicious Installation of Zenmap\n",
    "id: 5266a592-b793-11ea-b3de-0242ac130004\n",
    "description: Detects suspicious installation of Zenmap\n",
    "references:\n",
    "    - https://rmusser.net/docs/ATT&CK-Stuff/ATT&CK/Discovery.html\n",
    "author: Alexander Jaeger\n",
    "date: 2020/06/26\n",
    "modified: 2021/01/01\n",
    "logsource:\n",
    "    product: linux\n",
    "    service: shell\n",
    "detection:\n",
    "    keywords:\n",
    "        # Generic suspicious commands\n",
    "        - '*apt-get install zmap*'\n",
    "    condition: keywords\n",
    "falsepositives:\n",
    "    - Unknown\n",
    "level: high\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rule_by_text = ts_client.parse_sigmarule_by_text(MOCK_SIGMA_RULE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rule_by_text.search_query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rule_by_text.references"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzer\n",
    "\n",
    "Timesketch also has a Sigma analyzer that will add labels to all matching events. To do so you need a Timeline object to run the analyzer. The Analyzer then will take all rules installed on the Timesketch instance and go over the Timeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "timelines = sketch.list_timelines()\n",
    "timeline = None\n",
    "for timeline_ in timelines:\n",
    "  if timeline_.name == 'sigma_events.csv':\n",
    "    timeline = timeline_\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = timeline.run_analyzer(analyzer_name='sigma', ignore_previous=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sketch.get_analyzer_status()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This should show something like:\n",
    "```\n",
    "[{'index': <timesketch_api_client.index.SearchIndex at 0x126a09820>,\n",
    "  'timeline_id': 1,\n",
    "  'session_id': 1,\n",
    "  'analyzer': 'sigma',\n",
    "  'results': 'Applied 1 tags\\n* lnx_susp_zenmap.yml: 1\\nProblematic rules:',\n",
    "  'status': 'DONE'}]\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  },
  "metadata": {
   "interpreter": {
    "hash": "2556566ac8fd219b8b3cb0c76c8e4f226085ddd0e18628d37e0282e6a68fc0ff"
   }
  },
  "orig_nbformat": 2,
  "vscode": {
   "interpreter": {
    "hash": "a34c381f30693eab9e91429498d23c5bee65cfe0188b27a9a25a7dd65b2078b0"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
